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The purpose of this study was to evaluate methods for reducing the training and 
memorization required to manage air traffic in mid-term, Next Generation Air 
Transportation System (NexGen) airspace. We contrasted the performance of controllers 
using a sector information display and NextGen automation tools while working with 
familiar and unfamiliar sectors. The airspace included five sectors from Oakland and Salt 
Lake City Centers configured as a “generic center” called “West High Center.” The 
Controller Information Tool was used to present essential information for managing these 
sectors. The Multi Aircraft Control System air traffic control simulator provided data link 
and conflict detection and resolution. There were five experienced air traffic controller 
participants. Each was familiar with one or two of the five sectors, but not the others. The 
participants rotated through all five sectors during the ten data collection runs. The results 
addressing workload, traffic management, and safety, as well as controller and observer 
comments, supported the generic sector concept. The unfamiliar sectors were comparable to 
the familiar sectors on all relevant measures. 


Nomenclature 

M = mean 

SD = standard deviation 

t = Student’s t-statistic 

p = probability 

I. Introduction 

In the current air traffic management system, sectors of airspace have unique designs, nomenclature, and 
associated procedures. This has evolved to accommodate navigation, traffic flow, and safety requirements. 
However, the result is that controllers need extensive training on sectors they manage, which can make the transfer 
of staff between areas or facilities difficult. 

This Next Generation Air Transportation System 1 (NextGen) research project focuses on exploring the concept 
of “generic” airspace that can be worked with less training. 2 This will allow flexible allocation of human resources 
based on system need, less constrained by operator knowledge limitations. It will also provide a basis for 
dynamically reconfigurable airspace and other innovations to increase capacity. 3,4,5 Controllers could manage a 
variety of sectors, adaptable airspace, or dynamic sectors as dictated by traffic requirements. A byproduct may be 
improved situation awareness, given fewer demands on cognitive resources. Factors that tend to create complexity in 
sectors, and hence workload, could also be identified and minimized. 

This work builds on research completed by MITRE in 2007. 6,7 A Universal High Altitude Airspace (UHAA) 
evaluation was conducted with 16 controllers managing four unfamiliar sectors (including one sector used in the 
present study). Information was provided for sector boundaries, special use airspace, sector numbers and altitudes, 
radio frequencies of adjacent sectors, and major navigational aids (NAVAIDS). The results supported the UHAA 
hypothesis that controllers could work unfamiliar, ultra-high altitude sectors in the current operational environment. 

Current Federal Aviation Administration (FAA) plans for generic sectors include them as part of a Mid-term, 
High Altitude Airspace Concept. 8 It is assumed for this concept that data link will be in use for transfer of 
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communication, clearance delivery, and exchange of route information. Conflict detection and resolution (CD&R) 
will also be available, and flights will follow four-dimensional trajectories (also known as Trajectory Based 
Operations or TBO). 


II. Background 

The intent of the generic airspace task is to create or apply sector design features, automation tools, information 
displays, and procedures that will enable controllers to easily adapt to changes in airspace. This could help them 
move between sectors or respond to alterations in sector size and shape. 

For the initial phase of our work, we created a Controller Information Tool (CIT) that provided the necessary 
sector and traffic flow information to enable a controller to manage an unfamiliar sector. This sought to reduce the 
time required to learn and adapt to the sector by facilitating familiarization, rather than by changing sector 
characteristics. The goal was not to develop a stand-alone tool, but to present the information needed by the 
controller to work traffic. 

Other methods for creating airspace that is easier to learn and use include modifying the sector (e.g., removing or 
simplifying features and data) and providing automation tools to simplify the controller’s tasks. These strategies are 
interrelated in that automation can have the effect of reducing information needs. For example, if data link is used 
for transfer of communication to the next sector, the controller does not have to use adjacent sector frequencies, and 
these are no longer part of the sector map data that must be memorized. 

For Phase 1 of this task, we developed a generic sector in Oakland Center (ZOA) airspace and used present-day 
automation and procedures, employing the Multi Aircraft Control System (MACS) air traffic control simulator and 
the CIT. For Phase 2, we expanded the airspace by adding a second sector and applied mid-term NextGen 
automation tools to create a future air traffic management environment. We modified the sector information based 
on user input and continued to provide it on a stand-alone display. 

The purpose of the Phase 3 experiment reported here was to conduct a more thorough test of whether controllers 
can manage unfamiliar sectors with an acceptable level of workload, efficiency, and safety, in a generic airspace 
environment that includes NextGen automation tools and specific sector data. This phase focused on testing a 
“center” composed of generic sectors in an environment of data link and CD&R. We used ZOA sectors 30, 33 and 
43, and Salt Lake City Center (ZLC) sectors 42 and 45. These formed a fictitious area of sectors in a notional “West 
High Center,”, with an altitude floor of 34,000 feet. Figure 1 shows a map of this center (which we called “ZHW”) 
in the context of a national high altitude airspace concept being developed by the FAA. 



Figure 1. West High Center (ZHW) in the context of the National Airspace System 
and an FAA high altitude airspace concept (shaded area). 
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We evaluated the generic airspace concept by rotating controllers through the five sectors. Each of the controller 
participants in the experiment was familiar with one or two of the sectors, but not the others, and each controller 
knew sectors the others did not know. This approach enabled us to test the hypothesis that generic sectors can be 
adequately managed by controllers who are not familiar with them. We compared data collected from the “familiar” 
sectors with those collected from the “unfamiliar” sectors to look for differences in workload, safety, acceptability, 
and other measures. 


III. Technical Approach 


A. Software and Equipment 

The revised CIT showed a map of ZHW, with selectable views of each sector in a two-dimensional presentation. 
Information was displayed on a 30-inch monitor, mounted above the controller’s main radar display. The same 
trackball that was used to access the radar display also controlled the CIT. (See Figure 2 for a screen image and 
Figure 3 for the display configuration.) 

The available CIT views included: 

1 . Basic sector map with boundaries. 

2. Sector numbers, frequencies, altitudes, and special use airspace information. 

3. Focations and names of fixes and NAVAIDs. 

4. Major traffic flows (showing routes, traffic direction, and exit altitudes). 

The Crew- Vehicle Systems Research Facility Air Traffic Control Faboratory was used to test the generic 
airspace concept. Focated in Building N257 at NASA Ames Research Center, this laboratory has ten air traffic 
control positions with FAA en route consoles and large format displays. The facility also has pseudo-pilot 
workstations and a communication system. It employs MACS to realistically simulate the FAA en route air traffic 
control Display System Replacement user interface. MACS was configured to provide controller-pilot data link 
communications for all clearances (with voice backup) and CD&R (with suggested resolutions). 



Figure 2. CIT screen for West High Center. 
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Figure 3. CIT and radar display configuration. 


B. Traffic Scenarios 

We created two traffic scenarios for this test, drawn from actual ZOA and ZLC flights. Traffic was run 
independently in each of the five sectors. Scenarios had comparable workloads and were adjusted to provide a 
moderate level of activity. (The same basic traffic scenario was used for each version, with changes in aircraft call 
signs, starting times, and positions to make them appear different.) Our goal was to design the traffic problems so 
that they had moderate traffic and complexity. Table 1 shows the Monitor Alert Parameter 9 (MAP) values for the 
actual FAA sectors compared with the peak traffic counts for each scenaario. (MAP values are an estimate of the 
maximum capacity for the sector.) Routes were designed with the assumption that all flights were global positioning 
system equipped and made best use of published waypoints and Q-routes as opposed to using the ground-based jet 
route structure. One additional scenario was created for training. 

Table 1. MAP and maximum scenario traffic values for each sector. 


Sector 

MAP Value 

Maximum 

Percentage 

30 

13 

11 

77% 

33 

18 

16 

61% 

43 

18 

14 

83% 

62 

19 

11 

79% 

65 

23 

13 

57% 


Aircraft tracks were initiated at the start of each scenario inside or outside of the sector boundaries. Traffic built 
to a moderate level within 20 minutes after the beginning of the scenario. Each scenario ran for 45 minutes. (See 
Figure 4 for the major traffic flows in the five sectors.) 
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ZHW62 


ZHW43 


ZHW30 


ZHW65 


ZHW33 


Figure 4. ZHW major traffic flows. (Sector numbers were changed for the study.) 

C. Participants 

Five recently retired (within three years) air traffic controllers acted as test subjects. While with the FA A, the 
controllers had worked one or two of the five sectors that were being tested, and were not familiar with the other 
sectors. (Please see the experimental design in Table 2 for details.) Participants had the necessary skills to manage 
the sectors. We used five pseudopilots and three ghost controllers to support the simulation. Five retired air traffic 
controllers acted as observers. All participants were paid for their time. 

D. Experimental Design 

Each controller worked one or two familiar sectors and three or four unfamiliar sectors. The order of data 
collection runs was counterbalanced to minimize the practice effect. (That is, if participants’ performance improved 
over time, it would equally benefit all sectors.) The trials were separated into two groups of five runs each. This 
allowed us to determine if a significant practice effect occurred in the first set of five trials, compared to the second 
five. 

E. Procedures 

The study was scheduled for four days during one week (December 14 to 18, 2009). The five controller 
participants trained for one day, and ran traffic problems for three and one-half days. Upon arriving, they received a 
briefing on the study and detailed information on the automation tools in MACS. This was followed by three 
practice scenarios at their “home” sectors to gain experience with the tools. After that, there was an introduction to 
the generic airspace concept. There was no further practice prior to the data collection runs the next day. 

For the first data collection run on the second day, each participant was assigned to a workstation and provided 
with a hands-on introduction to the CIT. They were then presented with a 45-minute traffic scenario. Then each 
controller rotated to a new workstation (and new sector). An expert observer followed each controller during the 
experiment. The participants followed the sequence of sectors in the experimental design in Table 2. 
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Table 2. Experimental design for the Phase 3 Study. (The column headers have participant numbers and 
familiar sectors for each controller. In the body of the table, red indicates familiar sectors. There were two 
blocks of five trials each.) 


Trial 

Cl 

(ZHW 43) 

C2 

(ZHW 43) 

C3 

(ZHW 30&33) 

C4 

(ZHW 62) 

C5 

(ZHW 65) 

1 

ZHW30 

ZHW 33 

ZHW 43 

ZHW 62 

ZHW 65 

2 

ZHW 33 

ZHW 43 

ZHW 62 

ZHW 65 

ZHW 30 

3 

ZHW 43 

ZHW 62 

ZHW 65 

ZHW 30 

ZHW 33 

4 

ZHW 62 

ZHW 65 

ZHW 30 

ZHW 33 

ZHW 43 

5 

ZHW 65 

ZHW 30 

ZHW 33 

ZHW 43 

ZHW 62 

6 

ZHW 30 

ZHW 33 

ZHW 43 

ZHW 62 

ZHW 65 

7 

ZHW 33 

ZHW 43 

ZHW 62 

ZHW 65 

ZHW 30 

8 

ZHW 43 

ZHW 62 

ZHW 65 

ZHW 30 

ZHW 33 

9 

ZHW 62 

ZHW 65 

ZHW 30 

ZHW 33 

ZHW 43 

10 

ZHW 65 

ZHW 30 

ZHW 33 

ZHW 43 

ZHW 62 


F. Data Collection 

The objective of the study was to assess differences in controller workload and other measures while working 
familiar versus unfamiliar sectors of generic airspace, using mid-term NextGen automation tools and the CIT. The 
hypothesis was that controllers would demonstrate acceptable levels of workload and other measures in unfamiliar 
sectors, as compared to familiar sectors. 

The following is a list of the dependent measures: 

1 . On-screen Workload Assessment Keypad (WAK) 10 input every five minutes. 

2. Post-run questionnaire (workload and other measures). 

3. Over-the-shoulder data collected by controller observers (including workload ratings). 

4. MACS metrics (operational errors, time/distance in sector, number of data link messages). 

5. Post-experiment questionnaire. 

6. Frequency of access to sector data. (The CIT display timed out every 90 seconds and all key presses to access 
data were recorded.) 

7. Post-experiment debriefing to collect additional information. 

IV. Results 

This section describes the results from the analysis of the above data. Statistical tests were carried out with the 
data, where feasible, at the p < .05 level. Only the statistically significant test results are reported. Error bars on the 
graphs represent the standard error of the mean. 

Comparisons were carried out between measures collected in the first and second blocks of trials (WAK and 
post-run questions). No notable or statistically significant differences were observed, and so the data for all runs 
were analyzed together. 

A. Workload 

Workload was collected from three sources: the WAK, post-run questionnaire, and observer ratings. Controller 
subjective judgments of workload were taken every five minutes using the WAK. A one to five scale appeared at the 
top of the main radar display, and the participant could use the trackball pointer or labeled keyboard keys to enter 
the value. A rating of one was defined to be very low workload (such as on a night shift in a low volume sector) and 
five was defined as maximum workload (when the controller would normally request the help of a D-side assistant). 
Average and peak workload were rated on a five point scale after each run on the post-run questionnaire, and 
observers made ratings of how hard they thought the controllers were working. Figure 5 shows the results from these 
three sources. These independent measures are in general agreement that the workload for unfamiliar sectors was not 
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much different from that of familiar sectors. Workload was generally in the low to moderate range for all measures. 
One controller’s ratings on the WAK and post-run questionnaire were uniformly low (mostly one out of five) and 
these data affected the averages, which would have otherwise been closer to the mid-range. 
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Figure 5. Comparison of workload ratings (1 = low workload and 5 = high workload). 

B. Post-run Questionnaire 

The post-run questionnaire also included items about safety, traffic management, etc. (Ratings represented the 
average level of agreement with the item statement with 1 = “Strongly Disagree,” 2 = “Somewhat Disagree,” 3 = 
“Neutral,” 4 = “Somewhat Agree,” and 5 = “Strongly Agree.”) 

Figure 6 is a summary of the post-run questionnaire data averaged for familiar and unfamiliar sectors. The average 
ratings on the first four scales suggest that traffic was manageable in both familiar and unfamiliar sectors, and that 
safety was not compromised. It appears that more familiarity with the sectors would have changed the way the 
controllers managed traffic. This was the only result that showed a statistically significant difference, with familiar 
(M= 1.8, *SD=0.57) and unfamiliar (M= 2.8, SD= 0.36), £(4)=-3.31, p=.03). Required information (presumably on the 
CIT) was also available and easy to find. 

C. MACS Metrics 

MACS data were collected for separation losses, time/distance in sector, and data link uplinks. There were nine 
losses of separation during the simulation, as shown in Table 3. (Several recorded conflicts were eliminated for 
various reasons, but we adopted a conservative approach, retaining events if there was any doubt.) We suspect that 
several errors were due to controllers “trying out” the conflict probe and resolution tools. When considered as a ratio 
of the number of runs, there was not much difference between the familiar and unfamiliar sectors. (There were 12 
trials for familiar sectors and 38 trials for unfamiliar sectors.) 

Table 3. Losses of separation. 



Count 

Ratio 

Familiar 

2 

0.17 

Unfamiliar 

7 

0.18 


The differences in number of data link uplinks or time/distance flown in sector between familiar and unfamiliar 
sectors were small and not statistically significant. 
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Felt in Control of Traffic 


Could Comfortably Handle Traffic 

Could Comfortably Handle More 
Traffic 

Level of Familiarity Did Not Adversly 
Affect Safety 

More Familiarity Would Have 
Changed How I Managed Traffic 

There Was Sufficient Information for 
Controlling Aircraft 

It Was Easy to Find Information to 
Control Traffic 



1 2 3 4 5 


Level of Agreement 


Familiar 

Unfamiliar 


Figure 6. Summary of post-run data. 


D. Access to Sector Data 

The CIT was used to present data about the sectors. Figure 7 shows the results for how often controllers accessed 
three different types of sector data, using the on-screen buttons. (Sector frequencies and numbers were on the same 
button.) We can safely assume (based on participant feedback) that the data item being accessed in the 
“Frequency/Sector” category was adjacent sector number for handoffs. (Frequencies were automatically transferred 
to the aircraft.) Sector numbers were accessed more often for unfamiliar sectors. This difference was statistically 
significant, with familiar (M=4.2, *SZ>=3.15) and unfamiliar (M=10.9, SD= 4.22), ^(4)=-4.41, p=.0l). Ground aids 
(e.g., fixes) were not accessed very often, but somewhat more frequently for unfamiliar sectors. Route information 
was viewed at about the same rate for both familiar and unfamiliar sectors. 



Figure 7. Average frequency of CIT usage. 
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E. Post-experiment Questionnaire 

This questionnaire included items about the change in familiarity for the familiar versus unfamiliar sectors. The 
controllers were asked how much more familiar they were with each sector after the end of the 10 simulation trials. 
Figure 8 shows that there was a substantial increase in familiarity with the unfamiliar sectors, but not (as would be 
expected) with the familiar sectors. The difference for the unfamiliar sectors is significant with familiar (M= 2.2, 
SD= 1.26) and unfamiliar (M= 4.0, SD= 0.88), ^(18)=-5.49,/>=.00). 



Figure 8. Average familiarity with sectors before and after the experiment. 

It also appears that the CIT was useful to the participants for both kinds of sectors, as shown in Figure 9. Figure 
10 includes opinions about the NextGen tools, including data link, conflict probe, and the suggested conflict 
resolutions. Data link and the conflict probe appear to have support, though the feedback regarding the conflict 
resolutions was not as positive, and there is more variation in the responses. One controller reported not using the 
conflict resolution capabilities at all. 


© 3 


Familiar 


Unfamiliar 


Figure 9. “The Controller Information Tool was useful in helping me to control traffic.” 
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Data link was effective in helping me to 
control traffic. 


The ten-minute conflict probe tool was useful. 


The conflict resolutions generated by the 
conflict probe tool were effective. 


I could trust the resolutions provided by the 
conflict probe tool. 


1 2 3 4 5 

Level of Agreement 


Figure 10. Average agreement ratings to questions about NextGen automation tools. 

F. Post-experiment Briefing 

1. Overall Concept 

In general, the controllers supported the generic sector concept, saying it was easy to move between familiar and 
unfamiliar sectors. One suggested that this was smoother during the second set of five scenario runs, since there had 
been time to practice on the unfamiliar sectors during the first five runs. There was a comment that Sector 33 had 
altitude restrictions to airports and this additional complexity would make it more difficult for it to be generic. 
Moving between more sectors than in the current work environment might reduce complacency and help keep 
controllers “on their toes.” There could also be job satisfaction benefits from a “change of scenery.” 

If controllers moved between a larger number of sectors than today, there could be a loss of “comfort level” as 
they started to work a less familiar sector, though the participants said the sector knowledge would come back 
quickly. Over time, controllers learn the “traps” in sectors. (Traps are where losses of separation may occur.) One 
problem working a less familiar sector might be handling emergencies. Information on nearby airports, etc., might 
not be readily available from memory. Under normal conditions, the controller would have time to look these things 
up. If more attention had to be paid to learning an unfamiliar sector, controllers might have less time to “help 
airplanes” with routing requests, etc. 

The participants thought that all high-altitude sectors could be candidates for generic airspace. Adding a briefing 
prior to starting work would help with orientation to an unfamiliar sector. There was some discussion of whether 
new controllers should be introduced to simpler high altitude or generic airspace first. On the other hand, it might be 
easier for experienced controllers to move between generic sectors. 

2. NextGen Tools 

The NextGen tools (data link and CD&R) helped the controllers to manage traffic, but the participants suggested 
that high altitude airspace is generally not very complex. Data link downlinks from the aircraft might augment the 
generic sector concept in that aircraft could initiate trajectory requests via data link. Some controllers wanted a faster 
data link user interface. However, others liked our “three step method” for sending data link messages, since it 
allowed for an “are you sure?” function. A data link history list was recommended. 

Some participants found the long-range conflict probe to be helpful. However, they had suggestions about 
improving the alerting and user interface. Controllers also liked the “what if’ capabilities of the MACS trial planner. 
Color coding to identify separate pairs of conflicts would be useful. 


10 

American Institute of Aeronautics and Astronautics 


The controllers suggested trying a conflict resolution setting of seven to nine miles so that they would have more 
of a buffer and would not have to watch the conflict solutions so carefully. (The setting was at the minimum of five 
miles for our study.) Controllers found some of the conflict resolutions to be inefficient and of limited use. Some of 
the resolutions did not make sense (e.g., turning an aircraft in front of the conflicting aircraft). Participants also 
wanted more altitude (as opposed to route) resolutions. Some way of keeping track of which conflicts had been 
“fixed” would be useful. The simulation pilots sometimes took a long time to respond to data link messages. 

3. Controller Information Tool 

The CIT was considered to be very helpful, but some controllers did not like the location of the screen above the 
main MACS display. Suggestions for improvements included adding weather information, arrival routings, 
simplified Letters of Agreement, a drawing tool, and low altitude boundaries. Participants said that the most useful 
sector information was the adjacent sector numbers, followed by the fixes and NAVAIDs. The controllers noted that 
they did not need the fixes very much since they could use the graphical trial planning tools in MACS. 

4. Observer Comments 

The observers suggested that whether a sector is familiar or unfamiliar is not so much the issue as is how much 
information the controller needs to learn and memorize to work traffic in a particular sector. In unfamiliar sectors, 
controllers tended to focus mostly on applying basic separation, whereas in familiar sectors they applied additional 
control techniques. The second set of five runs seemed easier for the controllers, since they had time to practice 
during the first five runs. 


V. Discussion 

The key results for evaluating the generic airspace concept include the three workload measures (WAK, post- 
run, and observer) and other data that address safety and the ability of the controllers to work in unfamiliar airspace. 
These findings support the hypothesis that controllers are able to manage unfamiliar airspace with an acceptable 
level of workload, perceptions of control, and safety, within the simulation environment employed in this study. 
These measures were not much different between unfamiliar and familiar sectors. 

Data from the post-experiment questionnaire showed that the participants’ level of familiarity with the sectors 
markedly improved by the end of the experiment. Ratings for the unfamiliar sectors after the study were less than 
one point (on the five point scale) lower that for the familiar sectors. Thus practice significantly improved 
familiarity, but not quite to the level of being equal to sectors the controllers had worked extensively during their 
careers. 

In the post-experiment discussion, controllers said that they found working the unfamiliar sectors easier during 
the second set of runs. They indicated that there were some differences in how they managed traffic depending upon 
their level of familiarity with the sectors. While they thought the operations were manageable and safe, they 
indicated that more familiarity would have changed how they handled traffic. Without a deeper level of experience, 
they might be less knowledgeable about areas where conflicts might occur and not be able to pay as much attention 
to responding to pilot requests. The observers noted that in unfamiliar sectors, the controllers appeared to focus on 
basic control techniques, whereas they applied additional strategies in familiar sectors. Interestingly, several of the 
controllers said that there could be benefits from moving between a larger set of sectors. It could provide more 
variety to the job and might even increase alertness. 

Although there were some losses of separation during the experiment, this did not definitely indicate that 
unfamiliar airspace was an issue. However, it is not clear why there were operational errors in this study, given that 
the workload was only moderate. (We were not able to make recordings of the radar screens due to technical 
constraints in the simulator.) The conflict rates, given the number of runs, were similar in both familiar and 
unfamiliar sectors. We had the conflict resolution parameters set for five miles or 1000 feet (the minimum 
separation standards). This did not leave much room for error if controllers were using the CD&R tools in various 
situations. 

There was little difference on MACS performance measures between familiar and unfamiliar sectors, including 
time aircraft spent in the sector, distance flown in sector, and number of data link uplinks. If there were differences 
in how the controllers managed traffic in familiar versus unfamiliar sectors, it was not detected by these metrics. 
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The CIT was consulted mainly for adjacent sector numbers that were needed to initiate handoffs, and for route 
information. Controllers accessed the sector number information more often in the unfamiliar sectors. They 
responded that there was sufficient information to control traffic, and that it was readily available. Participants 
commented that the CIT worked well, but some preferred to have the display located lower down, so they do not 
have to look up to read the data. They had a few suggestions for additional content to be added to the CIT. 

The NextGen tools we used for this study included data link for all clearances. There was also CD&R set for a 
ten-minute look ahead, and conflict resolutions on demand. The controllers generally liked the data link 
environment, and gave it high ratings in the final questionnaire. There was some indication that the user interface 
could be more efficient. The conflict probe was also deemed useful, though we noted several false alarms due to 
simulation software problems during the scenarios, sometimes in overtaking situations. The conflict resolutions 
obtained lower controller ratings. Participants commented that a setting of more than the five-mile limit we used 
would probably make the conflict resolutions seem more trustworthy. Resolutions sometimes did not make sense 
(e.g., turning one aircraft in front of the other). Controllers did not always use the conflict resolution tool. 

VI. Conclusions 

The purpose of this experiment was to test whether a generic airspace concept that includes providing controllers 
with important sector data and NextGen automation tools would help them manage unfamiliar airspace. In any 
future system that employs generic airspace, controllers would be trained and experienced on all of the sectors they 
would control, even if the number was higher than today. Placing our participants in front of unfamiliar sectors to 
which they only had a brief introduction was “unfair” in a sense, since they were not provided with any formal 
training or practice. The results we obtained, then, may be for a more difficult case than would be found in a future 
air traffic control environment, such as a high altitude facility. 

All of the indicators from this study that addressed workload, traffic management, and safety, as well as 
controller and observer comments, supported the concept. The unfamiliar sectors were comparable to the familiar 
sectors for all relevant measures. 

The airspace information presented for the controllers was an important component of the generic airspace 
concept. Data link seemed to reduce the need for controllers to access sector frequency information, and perhaps 
also NAVAIDS. It is probably safe to assume that data link facilitates the generic sector concept by reducing the 
amount of information that has to be memorized by the controller, though this needs to be confirmed by further 
research. 

It is not certain from this study whether CD&R is an essential enabler for the generic airspace concept. We 
suggest that the ten-minute conflict detection probe, which received positive ratings, is a good feature to include. It 
provides the controller, who may not have worked a generic sector for a while, with some backup or a “safety net” 
to provide alerts about conflicts in the areas of the sector where conflicts might occur. More work needs to be done 
to understand conflict resolution tools before we can be certain that they are needed to support generic airspace. 

The maximum traffic counts in our scenarios were below the MAP values for the actual sectors and this may 
have accounted for the moderate workload ratings. Sector complexity ratings made by observers were also in the 
moderate range. While the generic airspace concept seems to work well under these conditions, it is not known at 
this point how controllers would respond to higher traffic levels or unusual situations. This is where the NextGen 
tools might be more important to help reduce workload and control errors. 

Should the generic sector concept move forward, it is clear from this study that even a limited amount of prior 
exposure to an unfamiliar sector can have benefits. Our participants commented on this, and indicated that their 
familiarity increased substantially after only two sessions on unfamiliar sectors. Should controllers eventually 
manage a larger set of sectors than today, an opportunity to practice for a short period of time prior to starting work 
in a fresh area would be helpful. This could be done in a training room, where controllers who are about to work a 
sector they have not seen for a while, could view a pre-recorded briefing or orientation, and work a sample traffic 
problem. For generic airspace, there may be a benefit to assigning more highly skilled or adaptable controllers to the 
environment, in that they may be more successful. 
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Recommendations : 


1. Conduct further generic airspace research to investigate the robustness of the concept under higher traffic 
levels and in conditions where there are off nominal events, such as weather, emergencies, or automation 
failures. 

2. Add further conditions to the automation environment, including TBO and sector management (e.g., 
combining/decombining 11 ), which are part of the FAA’s Mid-term, High Altitude Airspace Concept. 

3 . Consider moving the data that are currently displayed on the CIT to the main radar display. 

4. Conduct further development on the data link user interface to improve efficiency and usability. 

5. Conduct further work on the CD&R tools to improve accuracy, reliability, and acceptance. 

Acknowledgments 

The author sincerely thanks the Generic Airspace Team including Mark Evans, Rod David , Jim Gibson, Vimmy 
Gujral, Jim Miller, Dan Peknik, Jacob Pfeiffer, Bill Preston, John Shih, Kendrick Wang, and Forrest West. Generic 
airspace research is part of NASA’s Dynamic Airspace Configuration Research Focus Area at the NASA Ames 
Research Center. It is funded by the NASA NextGen Concept and Technology Project and by the FAA ATO 
Planning, Research, and Technology Office. The author wishes to thank Parimal Koparkekar (NASA Ames), 
Shannon Zelinski (NASA Ames), and Anton Koros (William J. Hughes Technical Center) for their support. 

References 

^oint Planning and Development Office, “Next Generation Air Transportation System Integrated Plan,” [online publication] 
URL: http://www.ipdo.gov/library/NGATS vl 1204r.pdf [cited 24 August 2010]. 

2 Kopardekar, P., Bilimoria, K., and Sridhar, B., “Airspace Configuration Concepts for the Next Generation Air 
Transportation System,” Air Traffic Control Quarterly , Volume 16, Number 4, 2008. 

3 Brinton, C., and Pledgie, S. “Airspace Partitioning Using Flight Clustering and Computational Geometry,” 27th Digital 
Avionics System Conference, St. Paul, Minnesota, 2008. 

4 Xue, M., “Airspace Sector Redesign Based on Voronoi Diagrams,” AIAA Guidance, Navigation and Control Conference 
and Exhibit , Honolulu, Hawaii, 2008. 

5 Yousefi, A., Khorrami, B., Hoffman, R., and Hackney, B., ’’Enhanced Dynamic Airspace Configuration Algorithms and 
Concepts,” Metron Aviation Inc., Tech. Rep. Report No. 34N1 207-00 1-R0, December 2007. 

6 Kalbaugh, S. K., and Levin, P., “Universal High Altitude Airspace (UHAA) Mid-term Concept,” MITRE Corp., Ltr. F063- 
L07-075, September 2007. 

7 Levin, K. M., “Universal High Altitude Airspace (UHAA),” Presentation, February 2007. 

8 Federal Aviation Administration, “An Operational Concept for Mid-Term High Altitude (High Performance-Based) 
Airspace,” Version 4.17 (draft), William J. Hughes Technical Center, January 2010. 

9 Federal Aviation Administration, Order JO 7210.3W, ’’Section 7. Monitor Alert Parameter,” [online publication] URL: 
http://www.faa.gov/documentLibrary/media/Order/FAC.pdf [cited 24 August 2010]. 

10 Stein, E. S., “Air Traffic Controller Workload: An Examination of Workload Probe.” (DOT/FAA/CT-TN84/24). Atlantic 
City International Airport, NJ: Federal Aviation Administration Technical Center, 1985. 

n Bloem, M„ and Gupta, P., “Configuring Airspace Sectors with Approximate Dynamic Programming,” Proceedings of the 
27 th Annual Congress of the Aeronautical Sciences , Nice, France, 2010. 


13 

American Institute of Aeronautics and Astronautics 




Generic Airspace Concepts and Research 


Richard Mogford, Ph.D. 
and 

The Generic Airspace Team 


September 14, 2010 


i 


Overview 


Background 

Phases 1 and 2 Research 
Phase 3 Research 
Summary 



2 


Background 



• In the current en route system, controllers are checked 
out on only five to seven sectors 

- Significant time is required to learn airspace and 
procedures 

• The Next Generation Air Transportation System 
(NextGen) will introduce changes in airspace usage 
and design 

- Controllers may move between more sectors to work 
traffic, as determined by system needs 

- Dynamic re-sectorization may redefine airspace 
boundaries based on traffic flow, weather, etc. 

- New operational concepts will require different airspace 
structure and procedures 

• The current paradigm may limit NextGen concepts 

• How can we make airspace easier to learn and 
manage? 
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Management of Airspace 




Skills plus information 


4 



Generic Airspace Strategies 

Three approaches for making airspace easier to learn and 
manage: 

1 . Identify sector information needed by the controller and display it 
at the radar position 

2. Incorporate automation 

- Can eliminate or reduce the need for memorizing sector information 

3. Modify sector characteristics or topography 

- Change boundaries, shape, size, navigational references, labeling, 
etc. 

Prior research: Universal High Altitude Airspace study conducted by MITRE. 



Approach 

Created a “Controller Information Tool” (CIT) to assist 
controllers to manage unfamiliar sectors 

Met with Airspace and Procedures Office at Oakland 
Center (ZOA) to determine information needed by 
controllers 

- Sector boundaries 

- Special Use Airspace 

- Altitudes, frequencies, and sector numbers 

- Fixes and navigational aids 

- Typical traffic flows 

- Procedures and Letters of Agreement 

Chose Sector 33 in ZOA (high altitude) for initial generic 
airspace research 

Started development and testing 



Controller Information Tool 



7 


• », 



Sector Data 







Effects of NextGen 

Data Link 


Concepts: 
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Controller Data Link User Interface 


Data link is used to transfer aircraft to next sector frequency. 
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Controller no longer needs to remember adjacent sector frequencies. 
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Effects of NextGen Concepts: 
Graphical Route Changes 
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Graphics-based tools are used to communicate, visualize, and modify 

flight paths, using data link. 
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Navigational references no longer need to be memorized. 
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NextGen Concepts Support 
Generic Airspace 



• Outcomes 

- Lower workload, improved safety 

- Controllers have less requirement to use sector frequencies 
and navigational references 

- NextGen automation reduces the amount of data that the 
controller needs to learn, recall, and actively retain in 
memory 

- Also results in more predictable flight trajectories (Trajectory 
Based Operations [TBO] or 4D trajectories) 
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Phases 1 and 2 Results 



• Basic testing completed 

• Generic sectors initially seemed to be feasible 

- Required sector information can be provided on a work aid 
or on an existing display 

- NextGen automation tools simplify information needs 

- Generic sectors may be practical in higher-complexity 
airspace than originally expected (e.g., ZOA33) 

• Further research 

- Start formal testing 

- Support FAA’s Mid-term, High Altitude, Trajectory-based 
Airspace Concept 
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FAA High Altitude Concept 
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• The base of high altitude airspace may vary across the NAS 

• All generic sectors are contained within high altitude airspace 

• TBO will be introduced first in high altitude airspace 

• Access to high altitude airspace will be restricted to equipped aircraft 
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Phase 3 Experiment 



• Human-in-the-loop experiment 

- Conducted week of December 14, 2009 

- Research goals: 

• Determine whether controllers can manage unfamiliar sectors 
with an acceptable level of efficiency and safety, as compared 
to familiar sectors 

• Provide data to calibrate dynamic density metrics for generic 
sectors 

• Test environment 

- Created a “West High Center” which included ZOA sectors 
30, 33 and 43, and Salt Lake (ZLC) sectors 42 and 45 

- Data link for transfer of communication and for sending 
route changes and clearances 

- Manual conflict detection and resolution (CD&R) for 
managing separation 

- Presented the information needed by the controllers to 
manage each of the five sectors 
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“West High Center” Airspace 

Map from MITRE UHAA Concept (9/07) 




Five contiguous sectors were treated as a subset of a notional high 

altitude facility (West High Center - ZHW). 
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Phase 3 Sectors 

Major Traffic Flows 
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Phase 3 Sector Data 
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Single Sector View 




23 



Laboratory Layout 
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Method 



• Experimental design 

- Five retired controllers, each familiar with one or two 
sectors, but not the others 

- Each controller worked all five sectors twice 

- Normal traffic conditions 

• Counterbalanced runs to reduce practice effect 

• Procedures 

- Participants were first given an orientation to the study 

- Then trained on automation tools with three runs on low to 
medium traffic 

- Given an overview of operations in the West High Center 

- Introduced to CIT 

- Then ran 10 data collection trials 
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Method 


• Data collection 

- On-screen Workload Assessment Keypad (WAK) input 
every five minutes 

- Frequency of access to sector data 

- Post-run questionnaire (workload and other measures). 

- Multi Aircraft Control System metrics (operational errors, 
etc.) 

- Post-experiment questionnaire 

- Over-the-shoulder data collected by controller observers 
(including workload ratings) 

- Post-experiment debriefing to collect additional information 

- Data to calibrate dynamic density measures 
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Results 

• Generic airspace concept seemed to be feasible (within 
study conditions) 

- Controllers found it acceptable to manage unfamiliar 
sectors, with the assistance of NextGen tools and important 
sector data 

- Observers agreed with this finding 

• Data supported these findings 

- Workload measures collected every five minutes and after 
each run showed small workload differences between 
familiar and unfamiliar sectors 

- Participants reported feeling in control of traffic and that 
safety was maintained 

- Found the sector data presented to be very helpful 

- There were nine losses of separation during the simulation, 
about equivalent between familiar and unfamiliar sectors 
(0.17 versus 0.18 conflicts per run) 
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Results 
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Comparable workload ratings for familiar and unfamiliar sectors. 
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Results 


Felt in Control of Traffic 

Could Comfortably Handle Traffic 

Could Comfortably Handle More 
Traffic 

Level of Familiarity Did Not Adversly 
Affect Safety 



Familiar 
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Changed How I Managed Traffic " 

There Was Sufficient Information for * 

Controlling Aircraft ** 

It Was Easy to Find Information to ■ — 

Control Traffic 


1 2 3 4 5 

Level of Agreement 


Acceptable ratings for both familiar and unfamiliar sectors. 

^ = statistically significant difference 



Results 


1 ^^ 



Familiarity with sectors before and after simulation. 
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Number of Times Accessed per Run 


Results 
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Access to sector data. 
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Results 



Data link was effective in helping me to t 

control traffic. 

The ten-minute conflict probe tool was useful. • • 
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Level of Agreement 


Ratings for NextGen tools. 
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Summary 

• Participant comments: 

- Supported generic airspace concept 

- Larger variety of sectors to work could improve job 
satisfaction and reduce complacency 

- Less familiarity might result in more basic control strategies 
and reduced service to aircraft 

- Some practice on an unfamiliar sector would be helpful prior 
to working traffic 

• Generic sectors may be feasible assuming: 

- High altitude, low to moderate complexity airspace 

- Important sector data provided to controller 

- Mid-term NextGen tools to assist controller and reduce 
sector information requirements 
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Recommendations 


• Conduct further generic airspace research to investigate 
the robustness of the concept under higher traffic levels 

• Explore conditions where there are off-nominal events, 
such as weather, emergencies, or automation failures 

• Consider best location for the sector data 

• Add capabilities to the automation environment, including 
TBO and sector management (e.g., combining/ 
decombining sectors) 
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